Automated classification of three-dimensional reconstructions of coral reefs using convolutional neural networks

© The Author(s), 2020. This article is distributed under the terms of the Creative Commons Attribution License. The definitive version was published in Hopkinson, B. M., King, A. C., Owen, D. P., Johnson-Roberson, M., Long, M. H., & Bhandarkar, S. M. Automated classification of three-dimensional reconstructions of coral reefs using convolutional neural networks. PLoS One, 15(3), (2020): e0230671, doi: 10.1371/journal.pone.0230671.Coral reefs are biologically diverse and structurally complex ecosystems, which have been severally affected by human actions. Consequently, there is a need for rapid ecological assessment of coral reefs, but current approaches require time consuming manual analysis, either during a dive survey or on images collected during a survey. Reef structural complexity is essential for ecological function but is challenging to measure and often relegated to simple metrics such as rugosity. Recent advances in computer vision and machine learning offer the potential to alleviate some of these limitations. We developed an approach to automatically classify 3D reconstructions of reef sections and assessed the accuracy of this approach. 3D reconstructions of reef sections were generated using commercial Structure-from-Motion software with images extracted from video surveys. To generate a 3D classified map, locations on the 3D reconstruction were mapped back into the original images to extract multiple views of the location. Several approaches were tested to merge information from multiple views of a point into a single classification, all of which used convolutional neural networks to classify or extract features from the images, but differ in the strategy employed for merging information. Approaches to merging information entailed voting, probability averaging, and a learned neural-network layer. All approaches performed similarly achieving overall classification accuracies of ~96% and >90% accuracy on most classes. With this high classification accuracy, these approaches are suitable for many ecological applications.This study was funded by grants from the Alfred P. Sloan Foundation (BMH, BR2014-049; https://sloan.org), and the National Science Foundation (MHL, OCE-1657727; https://www.nsf.gov). The funders had no role in the study design, data collection and analysis, decision to publish, or preparation of the manuscript

Matching Disparate Image Pairs Using Shape-Aware ConvNets

An end-to-end trainable ConvNet architecture, that learns to harness the power of shape representation for matching disparate image pairs, is proposed. Disparate image pairs are deemed those that exhibit strong affine variations in scale, viewpoint and projection parameters accompanied by the presence of partial or complete occlusion of objects and extreme variations in ambient illumination. Under these challenging conditions, neither local nor global feature-based image matching methods, when used in isolation, have been observed to be effective. The proposed correspondence determination scheme for matching disparate images exploits high-level shape cues that are derived from low-level local feature descriptors, thus combining the best of both worlds. A graph-based representation for the disparate image pair is generated by constructing an affinity matrix that embeds the distances between feature points in two images, thus modeling the correspondence determination problem as one of graph matching. The eigenspectrum of the affinity matrix, i.e., the learned global shape representation, is then used to further regress the transformation or homography that defines the correspondence between the source image and target image. The proposed scheme is shown to yield state-of-the-art results for both, coarse-level shape matching as well as fine point-wise correspondence determination.Comment: First two authors contributed equally, to Appear in the IEEE Winter Conference on Applications of Computer Vision (WACV) 201

Analysis of surface folding patterns of diccols using the GPU-Optimized geodesic field estimate

Localization of cortical regions of interests (ROIs) in the human brain via analysis of Diffusion Tensor Imaging (DTI) data plays a pivotal role in basic and clinical neuroscience. In recent studies, 358 common cortical landmarks in the human brain, termed as Dense Indi- vidualized and Common Connectivity-based Cortical Landmarks (DICCCOLs), have been identified. Each of these DICCCOL sites has been observed to possess fiber connection patterns that are consistent across individuals and populations and can be regarded as predictive of brain function. However, the regularity and variability of the cortical surface fold patterns at these DICCCOL sites have, thus far, not been investigated. This paper presents a novel approach, based on intrinsic surface geometry, for quantitative analysis of the regularity and variability of the cortical surface folding patterns with respect to the structural neural connectivity of the human brain. In particular, the Geodesic Field Estimate (GFE) is used to infer the relationship between the structural and connectional DTI features and the complex surface geometry of the human brain. A parallel algorithm, well suited for implementation on Graphics Processing Units (GPUs), is also proposed for efficient computation of the shortest geodesic paths between all cortical surface point pairs. Based on experimental results, a mathematical model for the morphological variability and regularity of the cortical folding patterns in the vicinity of the DICCCOL sites is proposed. It is envisioned that this model could be potentially applied in several human brain image registration and brain mapping applications

Biharmonic density estimate - a scale space signature for deformable surfaces

A novel intrinsic geometric scale space formulation for 3D deformable surfaces termed as the Biharmonic Density Estimate (BDE) is proposed. The proposed BDE signature allows for multiscale surface feature-based representation of deformable 3D shapes for subsequent image and scene analysis. It is shown to provide an underlying theoretical framework for the concept of intrinsic geometric scale space, resulting in a highly descriptive characterization of both, the local surface structure and the global metric of the 3D shape. The compactness and robustness of the proposed BDE signature are demonstrated via a series of experiments and a key components detection application